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Abstract — We study the repair problem of distributed 
storage systems in erasure networks where the packets 
transmitted from surviving nodes to the new node might 
be lost. The fundamental storage-bandwidth tradeoff is 
calculated by multicasting analysis in erasure networks. 
The optimal tradeoff bound can be asymptotically achieved 
when the number of transmission (packets) goes to infinity. 
For a limited number of transmission, we study the 
probability of successful regenerating. Then, we investigate 
two approaches of increasing the probability of successful 
regenerating, namely, by connecting more surviving nodes 
or by increasing the storage space of nodes. Using more 
nodes may pose larger delay and in certain situation 
it might not be possible to connect to more nodes too. 
We show that in addition to reducing repair bandwidth, 
increasing storage space can also increase reliability for 
repair. 

Index Terms — Network coding, Distributed Storage Systems, 
Erasure Channels. 

I. Introduction 

Disttibuted storage systems have atttacted substantial re- 
search interests recently for increasing applications in e.g., 
file sharing, cloud storage. The main benefit of distributing 
a file within a network is to provide reliable storage using 
unreliable storage nodes. Distributed storage systems also 
bring applications in wireless networks where links between 
storage nodes may be unreliable. For instance, consider an 
application of disttibuted storage systems in a delay tolerant 
network (DTN) with wireless channels, as shown in Fig. Q] 
A DTN is a kind of networks in which there might not be 
direct links between a source and destinations and services 
can tolerate an acceptable level of incurred delay []T|. In Fig. 
Q] a base station distributes a source file at the mobile nodes 
which may travel towards arbitrary directions. Then a data 
collector (DC) by meeting a certain number of these mobile 
storage nodes can rebuild the source file. 

Unreliability in distributed storage systems might stem 
from disk failure, but it is not limited to this reason. For 
example, in the network in Fig. Q] a node failure may be 
because a mobile node leave the system. In distributed storage 
systems in wireless networks, unreliability may also occur in a 
transmission process due to link failure, congestion, or buffer 
overflow and so on. We model an unreliable link in the network 





Fig. 1. A distributed storage system in a wireless network. Base station 
distributes a file among storage nodes on its coverage area. A data collector 
(DC) can recover the source file, even though it is out of base station coverage, 
after meeting certain number of storages node. In this example, the source 
file is coded by (n = 4, k = 2)-MDS code. Thus, the DC by downloading 
from any two storage nodes can rebuild the source file. 



to an erasure channel. When a packet erasure happens, all the 
information in the packet is lost. 

In a distributed storage system, a source file is generally 
encoded by erasure codes and distributed on redundant nodes. 
Thus, the file is still available even though some storage 
nodes fail. The maximum reliability by using erasure codes 
is achieved when a file contains k fragments and is encoded 
to n fragments, and every k out of n fragments can rebuild 
the original file. The codes are known as MDS (Maximum 
Distance Separable) codes. Then, an (n, k) erasure code has 
the MDS property if every k out of n fragments can rebuild 
the original file. For instance, if a file of size M is divided into 
k fragments and encoded by (n, k) MDS ((4, 2) here) codes 
on the application of Fig. Q] then a receiver (or data collector) 
by observing at least k, (k = 2) storage nodes can rebuild the 
source file. 

To maintain the reliability of the system, when a node fails, 
a new node should be regenerated. In the regenerating process, 
surviving nodes transmit sufficient data to the new node such 
that the new node can maintain the MDS property, but the 
new node may have different coded symbols. This is called 
the functional repair. We only consider the functional repair 
in the paper. A naive approach for repair is to rebuild the 
source file (by download M fragments from surviving nodes) 
and then regenerate the new node with the MDS property. 
However, downloading M fragments for regenerating M/k 



fragments is not efficient in term of bandwidth. Reference lHJ 
introduces network coding to distributed stroage systems in 
error- free networks (lossless network) and models the repair 
process by an information flow graph into a multicast problem. 
Moreover, it is shown that by slightly increasing node storage, 
the bandwidth can be reduced considerably 0. In J2], the 
fundamental bandwidth-storage tradeoff was established. The 
codes on the optimum tradeoff are called regenerating codes. 

Two extreme scenarios on the fundamental bandwidth- 
storage tradeoff are minimum storage regenerating (MSR) and 
minimum bandwidth regenerating (MBR) codes. For the MSR 
code, like the MDS codes, a file of size M is divided into 
k fragments and then encoded to n fragments. Hence, each 
node stores M/k fragments. Unlike MDS codes, MSR codes 
have the minimum bandwidth in regenerating a new node. 
That is, for d number of surviving nodes joining in the repair 
process, the minimum bandwidth by each surviving node (/3) 
is calculated by 

a m sr = M/k, 

Pmsr = k{d M k+1) ■ (1) 

For MBR codes, each node stores more fragments than M/k 
but used repair bandwidth is exactly the same as node storage, 
and is calculated by 

_ 2dM 
OCMBR - fc(23=fc+l)> 

PmBR= fe(2 j-fc+1) ■ ( 2 ) 

After seminal work in J2j, references [U] studies the code 
construction and achievablity of the functional and exact 
repair. In JU, cooperative regenerating codes are considered to 
reduce the bandwidth in the scenario of multiple-node failure. 
Reference [5| suggests surviving nodes cooperation in order 
to minimizes the cost of repair in multi-hop networks. Yet, in 
most of previous work of regenerating codes, it is assumed 
that links between storage nodes are perfect, without any 
error and erasure. In distributed storage systems especially 
in the wireless networks, as the example in Fig. 1, packets 
on the channels might get lost. Then, the redundant data 
needs to be transmitted for repair. Recently [6 1 has suggested 
a regenerating code which is resistant to a specific number 
of path failures by requesting more nodes to join the repair 
process. Particularly, in their approach the code resistant to t 
number of path failure, it is required to transmit from d = d+t 
surviving nodes instead of d nodes (that for perfect channels). 
However, reference [6] has not studied the probability of 
successful regenerating and how to construct the optimal (in 
regard of probability of successful regenerating) codes. We 
call a regenerating process is successful when the new node 
beside surviving nodes has MDS property. We shall consider 
the probability of successful regenerating in the repair process 
and show that finding the optimum values for d, and d 
depends on the erasure probability of the links. Thus, we 
can find the code maximizing the probability of successful 
regenerating, given the constraints of bandwidth and storage 
capacity. We also find that by using slightly more storage, the 
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Fig. 2. Fundamental bandwidth- storage tradeoff in erasure networks, n = 
10, k = 5. Higher link failure probability (e) poses higher traffic on the repair. 

probability of successful regenerating can be further increased. 
This approach suits for applications when requesting more 
surviving storage nodes may cause large delay in the repair or 
when there is not possible to connect to more storage nodes. 

Other related works on network coding for erasure networks 
are as follows. The capacity of wireless erasure networks has 
been studied in Q. In references |0, 10, the probability 
of successful reconstruction of a source file is studied in 
erasure networks. However, the papers do not studied the 
probability of successful regenerating. References |Q], Q8j 
study approaches in delay tolerant networks. In [8 1, the author 
show that there is no unique answer to the question whether 
coding for distributing a file maximizes the probability of 
successful reconstruction or not. 

The rest of this paper is orangized as follows. In Section 
HI1 we study the fundamental bandwidth-storage tradeoff in 
erasure networks. In Section [III] we study the maximum prob- 
ability of successful regenerating. In Section|IV]we propose to 
use more storage for increasing the probability of successful 
repair. Finally we conclude the paper in Section [V] 

II. Fundamental bandwidth-storage tradeoff in 

DISTRIBUTED STORAGE NETWORKS WITH ERASURE 
CHANNELS 

In O the fundamental bandwidth-storage tradeoff was 
found by cut analysis in a multicast network where links 
have no error or erasure (lossless networks). For the multicast 
problem in erasure networks, the capacity of networks was 
found in Q. Reference [7| proved the multicast capacity can 
be achieved in erasure networks by linear network coding. 
Using similar approaches to (2), Q, we can find the fun- 
damental tradeoff of distributed storage systems in erasure 
networks. To have a closed-form formula for the fundamental 
bandwidth-storage tradeoff with erasure channels, we consider 
the scenario when all the links in the network have equal 
erasure probabilities e. Then, we formulate the tradeoff as 
follows. 

Proposition 1: Consider a repair process in a distributed 
storage system where channels from d surviving nodes (d < 
(n — 1)) to the new node have an erasure probability e. 
For any a > a*(M,n,k,d,j,e) there is a linear network 



code to achieve the point (n, k, d, a, 7, e), and for a < 
a*(M,n,k,d,j,e), it is impossible to find a code. Here n 
denotes the number of storage nodes, and every k storage 
nodes can rebuild the source file, d is the number of surviving 
nodes participating the repair, and a is each node storage, 
and 7 is total repair bandwidth. Therefore, the fundamental 
bandwidth-storage tradeoff can be calculated as, 



Probability of successful regenerating in a DSS with n=l(), k=5 



M/k 

M-g(i)(l-p)i 
k — i 



where, 



and, 



2Md 



(2k- i- l)i + 2k(d-k + 1)' 
(2d + 2k + i + l)i 



9(i) = 



2d 



(3) 



(4) 



(5) 



Proof: The proof is similar to that in [2|. The only 
difference is that in the erasure network the new node in 
average receives (1 — e)f3 from each surviving node. ■ 
Fig. 12 shows the fundamental bandwidth-storage tradeoffs 
when links have erasure probabilities e = 0.1, 0.2, 0.3, respec- 
tively. As we expect, larger erasure probabilities lead to higher 
traffic in the repair process. These optimum bandwidth-storage 
tradeoff can be achieved asymptotically, i.e., when the number 
of transmission packets goes to infinity. When feedback is 
available on the network the optimal bound can be achieved 
by retransmission (potentially infinite). However, for a network 
without feedback, rateless random linear network codes ifPTl 
can asymptotically achieve the optimal tradeoff. For a limited 
number of transmission (retransmission) we are interested to 
maximize the probability of successful repair. We note that a 
repair process is called of success when the new node has the 
MDS property (along with surviving nodes). 

III. Maximizing the probability of successful 

REGENERATING p s 

Obviously, one approach to provide redundancy (for era- 
sure) for repair is to connect more surviving nodes to the new 
node. That is, for the repair in a lossless network 7 = d/3 
packets are transmitted from d nodes (each node transmits /3 
fragments). Then in an easure network, d (d > d) nodes 
transmitting 7 = d /3 packets may be necessary. One inter- 
esting scenario is that with the constraint of the total repairing 
bandwidth in the network (7 < jth), how to find the optimal 
repair (maximizing the probability of successful regenerating 
p s ). We will show that solving the problem depends on the 
link erasure probability. 

To formulate the problem, consider a repair process in a 
distributed storage system. Links from surviving nodes to the 
new node fails with probability e. To simplify illustration, we 
assume all the links have equal erasure probabilities. To com- 
bat link erasures, the redundant data should be transmitted. In a 
lossless network for regenerating a new node with parameters 
(n,k,d,a,j = d(3). By receiving 7 = df3 data, the new 
node is regenerated successfully. Therefore, a regenerating 
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Fig. 3. A scheme with a higher probability of success in one region might 
has a smaller probability of success in other region. 



process is successful when all 7 = dp data are received. 
However, for an erasure networks, failure at links can cause 
the repair unsuccessful. Thus, to increase the probability of 
successful regenerating, one approache is to use d (d > d) 
surviving nodes for repair. Each surviving node still transmits 
/? fragments of data. Hence, the probability of successful 
regenerating is that the new node receives from at least d out 
of d surviving nodes, which is calculated by 
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In Fig. [5J the probability of successful regenerating with 
various erasure probabilities using two regenerating schemes 
has been shown. In the example, the distributed storage system 
contains n = 10 storage nodes and every k — 5 nodes 
can rebuild the source file. In the first scheme a minimum- 
bandwidth regenerating (MBR) code with parameters d = 
7,d = 9 is used. In the second scheme a minimum-bandwidth 
regenerating code with parameters d — 5,d — 6 is used. 
Using ©, the transmitted bandwidth for the former is 7 = 3.6 
and for the latter is 7 — A. From Fig. [3] we can see that even 
though the bandwidth of the second scheme is higher than the 
first one, it has lower probabilities of success for e < 0.4. 
This example motivates us to further investigate the problem 
of maximizing the probability of successful regenerating for 
the given repair bandwidth. 

Given the constraint on the transmitted bandwidth (7 = 
d j3), d and d can be optimized to maximize the probability of 
successful repair. The optimization problem can be formulated 
as follows, 



max p s 

d.d' 

s. t. d' (3 < 7 t/l 



(7) 



For illustration, we use the optimization problem in the 
previous example and find optimal d and d for given 7^ < 5 
over different erasure probabilities. The optimal d and d with 
different erasure probabilities have been shown in Fig. |4] We 
can see that for the network with high erasure probabilities, 
using fewer surviving nodes in the repair process maximizes 



Maximizing p s for different integer values of d and d 
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Fig. 4. For a network with low e (erasure probability), maximizing p s uses 
a larger number of surviving nodes for repair. For a network with high e, 
maximizing p s finds a smaller number of surviving nodes for repair. 

p s . Conversely, for the network with low erasure probability, 
using more surviving nodes maximizes p s . 

IV. Increasing storage space of nodes 

To provide redundancy, one approach is to use more sur- 
viving nodes for repair. However, connecting to more storage 
nodes may lead to larger delay. For instance, in Fig. Q] the new 
node may have to wait longer time to meet more nodes which 
are moving. Another approach of improving reliability is to 
retransmit the lost packets from the surviving nodes to the new 
node. However, this will lead to larger delay and also feedback 
may be necessary. In what follows, we propose to increase 
storing capacity of the nodes to increase the probability of 
successful regenerating. 

For illustration, consider an example of a distributed storage 
system with four nodes shown in Fig. [5] Assume M = 4, 
n = 4, k = 2, d = 3 and each node stores two fragments 
(a = 2). By (Q]), the optimal repair-bandwidth in a lossless 
network is j3 = 1. Then, when a node fails, the new node is 
regenerated by acquiring /3 = 1 fragment (linearly encoded 
from data stored in surviving node and denoted by fx, fi 
and /3 respectively) from each of three surviving nodes. We 
assume the channels with the erasure probability e = 0.1. 
We also assume no feedback to surviving nodes. Thus the 
retransmission of fragment fi gives the probability of success 
Ps = (ELi(! ~ e) i e (2_t) )(l - e) 2 = 0.8019. Yet if a 
linear combination of /i,/2, and can be stored in node 
1, as shown in Fig. [6] then the probability of success is 
Ps = ELsC 1 - £) l E (4 ~° = 0.9477 (since any 3 out of 
4 transmitted fragments can regenerate the new node). Note 
that two schemes use the same bandwidth for repair. The 
example shows that increasing storage space can increase the 
probability of sucessful repair. In the following, we formulate 
this problem in a general setting. 

Consider the repair process in a distributed storage system 
where each node stores a = ot\ + a 2 fragments. Assume 
kotx fragments from k nodes can rebuild the original file. 
If channels are error-free then a new node is regenerated by 
transmitting fa fragments encoded from a\ stored data of a 
node. To increase the probability of successful regenerating, 




Fig. 5. Retransmission is used to recover the lost packet. 




Fig. 6. Larger storage capacity increases the probability of successful 
regeneration than retransmission. 

each surviving node transmits fa additional fragments to the 
new node. To simplify cut analysis on the network, we split a% 
and «2 storage fragments of each node, as shown in Fig. [7] (it 
can be proved that there is no loss in separating a\ and a 2 )■ 
Note that fa fragments are encoded from ot\ stored fragments, 
and fa fragments are encoded from other an stored fragments. 
In total, dpi + dfo fragments are transmitted from surviving 
nodes. However, due to link failures, the new node receives 
d\(3\ and d^h fragments where g?i,c?2 < d. We shall show 
how to calculate p s from di,d2, fa as follows. 

Proposition 2: To regenerate a new node, which receives 
from d storage nodes by fa fragments from cti fragments of 
each surviving node and fa fragments from other «2 fragments 
of the node, then the new node can be regenerated by linear 
network coding if difa + dnfa + 2~2i=i Tnin(ati, (d — i)fa) > 
M. Here d% and c?2 are the number of links in the network for 
transmitting fa and fa fragments without erasure, respectively. 

Then, the probability of successful repair is, 

Ps = EE (, d )(, d )(i- £ ) di+d2 £ 2d - (tii+d2) /(di,d 2 ) 

d 1= 0d 2 =0 V 1/ V 2/ 

(8) 

where, 

r i if dtfa + d 2 fa 

I(d 1 ,d 2 ) = I > M — Eti 1 min(ai, (d - i)fa), (9) 
[ otherwise 

Proof: Own to space limitation, we skip the proof here. 




Fig. 7. Cut analysis for the network with additional storage space 012 which can increase p s . 
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Fig. 8. The filled blue area is for the MSR code providing a higher probability 
of successful regenerating than the MBR code. 

By the Proposition [2] we can find the optimal storage 
allocation to achieve the maximum probability of successful 
regenerating for a given storage space and repair bandwidth. 
More formally, assume that storage space per node is bounded 
by ath and the maximum possible total repair-bandwidth is 
jth- Then the optimization storage allocation is formulated 
as, 

max p s 

s. t. ai + a 2 < a th ^Q) 

d0i + dfo < lth 
fi 2 < a 2 - 

We use an example for illustration, which finds the optimal 
storage allocation between MSR and MBR codes in a network 
with n = 10, k = 5 and e — 0.1. We know that MSR codes 
use less storage, but they need larger repair-bandwidth. On the 
other hand, MBR codes use more storage at each storage node, 
but they have lower repair-bandwidth. We use the storage and 
bandwidth as variables, as shown in Fig. [8] Then we observe 
that for low storage space, the optimal solution is to use MSR 
codes. It is interesting to see that for the large storage and 



low bandwidth region, the optimal solution is still to use MSR 
codes. Our investigation shows that MBR codes lead to higher 
probability of successful regeneration only when there is high 
storage space and large bandwidth available. 

V. Conclusions 

We study the regeneration problem for distributed storage 
systems when the channels are unreliable. We show the funda- 
mental storage-bandwidth tradeoff for erasure networks. Then 
the probability of successful regenerating is analyzed. We use 
two approaches to maximize the probability of successful 
regenerating. The first is to use more surviving nodes for 
repair. The second is to increase the storage space of storing 
nodes. We show that the optimal solution relies on the channel- 
erasure probabilities. We show that increasing storage space 
can also increase reliability for repair. 
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